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ABSTRACT 



This study is the final report of an investigation into the 
effectiveness of Student Teams Achievement Divisions (STAD) , a system of 
cooperative learning at the postsecondary level. The study specifically 
examined the relationships between a student's evaluation of his/her own 
effectiveness as a STAD team member, and other variables in a collaborative 
educational psychology course. Students (n=154) were assigned to eight 
heterogeneous teams which met together weekly to work on projects. At the end 
of the semester students rated both the course and themselves and their team 
members. The study found: (1) a substantial positive correlation between the 

student's self-rating and the rating given him/her by other team members; (2) 
a substantial positive correlation between the student's self-rating and the 
mean rating given to other team members; (3) a modest, positive correlation 
between the student's self-rating and the student's evaluation of the course; 
(4) a modest positive correlation between the student's self-rating and the 
student's evaluation of the teacher's motivational decisions; and (5) 
positive correlations between the student's self-rating and most of the 
student's formal grades. Appended are data tables, course rating scales, and 
team member performance appraisal forms. (Contains 18 references.) (DB) 
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CORRELATES OF SELF CONCEPT IN COLLABORATIVE LEARNING 
William J. Gnagey, Richard B. Sarles & Tracy R. Sarver 
Illinois State University 

Perspective 

This study is the eighth in a series of investigations of the effectiveness for imiversity 
students of Student Teams Achievement Divisions (STAD), one system of cooperative 
learning. These studies have been based on the research of Robert E. Slavin (1983, 
1989, 1990, 1991) who focused primarily on STAD's use in public schools. This system 
features small heterogeneous teams of 4-6 members who tutor each other on the 
material in the course and prepare each other for weekly quizzes that measure chapter 
objectives. Students take the quizzes individually, but are awarded bonus points on 
the basis of the team's mean quiz score. 

In our first experiment (Gnagey, 1988), four sections of sophomore educational 
psychology students (N=145) were divided into equivalent experimental and control 
groups. After a three week baseline period, both groups were taught in the STAD 
format for the rest of the semester. While students in the control group were awarded 
bonus points on the basis of the mean improvement of their teams, students in the 
experimental group received bonus points for their own individual improvement. 
Although no significant differences were foimd in the achievement of the two groups, 
the experimental classes rated the course as more effective than did the control classes. 

In our second experiment (Gnagey & Ostrowski, 1991), it was hoped that the social 
loafing of some team members could be prevented by publicizing their individual 
contributions to their teams. Two classes of educational psychology students (N=75) 
randomly assigned to control and experimental sections were taught in a conventional 
lecture-discussion format for a four week baseline period. Both the experimental and 
control sections were taught using the STAD approach for the remainder of the 
semester. The control class received anonymous feedback concerning their team’s 
quiz performance, whereas the experimental class received additional feedback 
making them privy to the scores of other members of their own teams. No significant 
differences were foimd between the mean quiz, midterm, and final exam scores for the 
two sections. 

In our third experiment (Gnagey & Ostrowski, 1992), we tried to determine the 
differential effects of two ways of awarding bonus points within the STAD format. 
Two forty-student adolescent development classes and two forty-student educational 
psychology classes served as subjects in parallel experiments. During the first half of 
the semester, all classes were taught in the same STAD format in which bonus points 
were awarded to all members of the three teams with the highest mean chapter quiz 
scores. During the last half of the semester, students in one randomly chosen 
adolescent development class and one randomly chosen educational psychology class 
were awarded bonus points on the basis of their team's mean improvement points. 
The award structure of the other two classes remained the same. For each experiment, 
the mean scores of the experimental and control groups were compared on seven 
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quizzes, a final examination, and the University Course Rating Scale. In each 
experiment, the "mean improvement points" class outscored the "mean raw score" 
class on only one chapter quiz. It was concluded that the independent variable was 
not robust enough to materially influence either students' achievement or their 
evaluation of the course. 

In our fourth experiment (Gnagey & Navarro, 1993), we compared the effects of two 
methods of constructing learning activities for classes being taught in the ST AD 
format. One hundred fifty-four students in four sections of educational psychology 
served as subjects. During the first half of the semester, all sections were taught in the 
same STAD format during which the instructor authored and assigned activities 
designed to assist team members in preparing each other for the weekly chapter 
quizzes. During the second half of the semester, two randomly chosen experimental 
classes prepared and carried out their own activities, while the two control sections 
continued to use those prepared by the instructor. Comparisons of the experimental 
and control group means on five chapter quizzes, the final examination, and the 
University Course Rating Scale revealed no significant differences on any of these 
measures of achievement or the course evaluation. It was concluded that after one-half 
a semester's experience with instructor-constructed learning activities, student teams 
were able to plan and carry out some that were at least as effective as those 
constructed by their teacher. 

In our fifth experiment (Gnagey & Navarro, 1994), we wanted to see if awarding 
bonus points for both mean quiz scores and mean improvement points would 
produce higher class achievement and course evaluation than either method by itself. 
One hundred fifty-four educational psychology students in four sections of 
educational psychology served as subjects in the experiment. At the beginning of the 
semester, all sections were taught in the usual STAD format with bonus points being 
awarded for correctly completing learning activities on Wednesdays, and for being in 
one of the four teams scoring highest on the weekly chapter quizzes. After Quiz 4, 
one section chosen at random began getting bonus awards for improvement scores 
while the other three sections continued being awarded points for high mean team 
quiz scores. After the midterm examination, two of these three teams were randomly 
assigned to other treatments. In one section, the two teams with the highest mean 
quiz scores and the two with the highest mean improvement scores were awarded 
bonus points. In the second, the four teams with the highest mean improvement 
points were rewarded. At the end of the semester, a final examination, and the 
University Course Rating Scale were administered to all sections. No significant 
differences were fovmd on any of these comparisons save for two chapter quizzes. It 
was concluded that none of the experimental variations had a systematic effect on 
either learning or course evaluation. 

. In our sixth experiment (Gnagey and Denoyer, 1995) we compared process and 
product methods of awarding bonus points to STAD teams. One hvmdred sixty-five 
students in four sections of educational psychology served as subjects in the 
experiment. They were assigned by the usual registration procedures employed by 
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the University and were mostly sophomores taking their first course in a program 
designed to produce high school teachers. 

For the first three weeks of the semester, all four classes were taught according to the 
usual STAD format. A General Linear Models Analysis indicated that there were no 
significant differences among the four sections on any of the first three quizzes. At 
this time, two experimental and two control groups were selected using a random 
number table. While the Wednesday teamwork and Friday quizzes remained the 
same for all sections, the teamwork for the experimental sections was graded as a 
project instead of being the basis for bonus points. Bonus points were awarded 
instead for mean quiz scores. In essence, the experimental subjects received bonus 
points for a learning product (quiz scores) while the control sections continued to be 
rewarded for a learning process (learning activities). T- tests were performed between 
the combined experimental and combined control group data for the remaining nine 
chapter quizzes, the midterm and final examinations, and the first and second 
administrations of the University Course Rating Scale (a course evaluation device) 
and the Team Member Performance Appraisal (a combined rating of the effectiveness 
of one's team members). Since no significant differences appeared in any of these 
analyses, it was concluded that teachers may award extrinsic reinforcers for either 
learning activities (the learning process) or quiz scores (the learning product) with 
equal results in the acquisition of subject matter, the evaluation of the course, and the 
attitudes developed among teammates. 

In our seventh experiment (Gnagey & Potter, 1996), we assessed the differential effects 
of a one-team and a two-team approach to the Slavin's STAD format for collaborative 
learning. Four sections of educational psychology students (N=169) were randomly 
divided into 2 experimental and 2 control classes after being taught for the first eight 
weeks of the semester in the usual STAD format. During the last eight weeks of the 
semester, the experimental classes were assigned to new STAD teams, while the 
control classes remained in the teams assigned to them in the beginning of the 
semester. The combined data from the two experimental and two control classes were 
compared at the end of the semester. No significant differences were found between 
the two groups on the chapter quizzes, the final examination, the University Course 
Rating Scale, and the Team Member Performance Appraisal. It was concluded that 
the formation of new groups at midterm with the additional record keeping involved, 
was probably a waste of the instructor's time . 

Objectives 

The objectives of the present investigation concern the relationships between a 
student's evaluation of his/her own effectiveness as a STAD team member, and 
several other variables in a collaborative educational psychology course. Since small 
group learning is central to the course, and since the operation of the team produces 
bonus points for each of its members, we expected to find positive relationships 
between measures of the effectiveness of that team, and measures of academic 
achievement and of instructor/course effectiveness. The following five hypotheses 
were tested; 
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1. There will be a significant positive correlation between a student's self-rating and 
his/her rating by the other team members. 

2. There will be a significant positive correlation between a student's self-rating and 
his/her rating of the other team members. 

3. There will be a significant positive correlation between a student's self-rating and 
his/her evaluation of the course. 

4. There will be a significant positive correlation between a student's self-rating and 
his/her evaluation of the instructor's motivational decisions. 

5. There will be a significant positive correlation between a student's self-rating and 
his/her achievement in the course. 

Methods and Techniques 

One hundred fifty-four students in four sections of educational psychology served as 
subjects in the experiment. They were assigned by the usual registration procedures 
employed by the University and were mostly sophomores taking their first course in a 
program designed to produce high school teachers. 

At the beginning of the second week of the semester, students were assigned to eight 
heterogeneous teams using a class list which had been sorted by scores on a thirty- 
item quiz over the first chapter in the text. The student with the highest score was 
placed on team one, the second highest on team two, the third highest on team three 
and so on until all eight teams had one member. The ninth student was placed in team 
eight, the tenth in team seven, and so on until all teams had two members. The 17th 
student was then placed in team one, and this reversal system continued until all 
students were assigned. The process was done for women first and then for men in 
order to balance each team for gender. 

For the remainder of the semester, all four classes were taught according to the usual 
STAD format. Each Monday, the assigned chapter was introduced and appropriate 
material was presented by way of lectures, films or videos. A list of key essay 
questions was distributed to all students with the promise that the multiple choice 
chapter quiz would measure their knowledge of the topics indicated by those 
questions. Each Wednesday, heterogeneous teams of 4-6 members convened to work 
on projects which involved the application of the text material to practical situations. 
Bonus points were awarded for accurate work. 

Each Friday after the corrected teamwork was returned, the class was divided into 
two activities. At the beginning of the hour, the instructor held a question and answer 
session in which students could ask for clarification of difficult concepts found in the 
key questions. When there were ho more questions, a 30-item multiple choice chapter 
quiz was administered to all students individually. At the end of the Friday class, the 




4 



6 



appropriate chapter assignment for the following week was written on the 
chalkboard. 

During class on the following Monday before the new chapter was introduced, all 
quizzes were returned, and a list of "grades so far" was posted by student social 
security numbers. Students were encouraged to write rebuttals, by Wednesday, to any 
of the items they missed but felt they should have credit for. The STAD cycle then 
began for the new chapter. 

At midterm, a 60-item multiple choice examination was administered over the first six 
chapters. During the second half of the semester, the same STAD format was followed 
for the remainder of the chapters that were assigned. 



At the end of the semester a 60-item multiple choice examination was administered to 
all four classes covering only those chapters assigned since midterm. The University 
Course Rating Scale (UCRS), the Team Member Performance Appraisal (TMPA), and 
the TARGETT Rating Scale were also administered at that time. 

The UCRS is composed of twelve items on which students are asked to evaluate all 
aspects of the course: clarity of objectives, projects and papers, textbook and other 
assigned readings, in-class activities, quizzes and exams, feedback, interestingness, 
instruction, grading procedures, grading fairness, amount learned, expected level of 
performance (standards), and group activities (See Appendix 1). 

In a previous study of 158 students, (Gnagey & Ostrowski, 1992), factor analysis of the 
UCRS (see appendix) using varimax rotation revealed three principal factors (See 
Table 1.). The first accounted for 31% of the variance and loaded heavily on items D 
(in-class activities), G (interestingness), H (instruction); J (amount learned), K ( 
expected level of performance), and (group work). The second principal factor 
accounted for 29% of the variance and loaded heavily on items A (clarity of 
objectives), B (out of class papers and projects), C (text), E (quizzes and exams), and I 
(fairness of grades). The third factor accounted for only 11% of the variance and 
loaded on item F (feedback). 

The test-retest coefficient of reliability for the UCRS total score was .81 using 48 
students in similar classes with one administration at midterm and the other eight 
weeks later during the finals. 

The Team. Mem.ber Performance Appraisal (TMPA) rating scale (See Appendix 2) was 
administered requiring all students to rate themselves and the other team members of 
their teams on the following criteria: a) team meeting attendance, b) contribution of 
ideas, c) completion of assignments, d) promoting positive feelings among team 
members, e) encouraging the expression of other team members. The ratings were 
done using a 5-point scale for each criterion: one indicating "almost never" and five 
indicating "almost always". Three variables were derived from these scores. TMPAl 
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was the mean rating a student gave the other team members; TMPA2 was the mean 
rating a student received from the other team members; and TMPAS was a student's 
rating of his/her own effectiveness as a member of the team. 

The TARGETT Rating Scale is based on a model organized by Epstein (1989) using six 
areas identified by Carol Ames (1990, 1992) where teachers make decisions that can 
influence student's motivation to learn. These are : (a) the nature of the learning task, 
(b) the autonomy students are allowed, (c) how students are recognized for their 
work, (d) how they are grouped, (e) how they are evaluated, and (f) how time is 
allotted for the work. Woolfolk (1995) added (g) teacher expectations as a seventh 
area. Students were asked to rate their educational psychology course on all seven 
criteria using the following five point scale: 5 = excellent; 4 = good; 3 = average; 2 = 
poor; 1 = unacceptable (See Appendix 3). 

Results and Conclusions 

Pearson Product Moment correlations were calculated between the student's self- 
rating as an effective STAD team member, and all other variables described in the 
hypotheses above (See Table I). 



Hypothesis 1 was confirmed. There was a substantial positive correlation between 
TMPAS, the student's self-rating, and TMPA2, the mean rating that other teaih 
members gave the student ( r=.4296, p.=.0001). This agreement between the student 
and his/her team mates suggests that the self-rating has some outside validity. 

Hypothesis 2 was confirmed. A substantial positive correlation was found between 
TMPAS, the student's self-rating, and TMPAl, the mean rating the student gave to 
other team members (r=.5279, p.=.0001). This finding suggests that the student sees 
effectiveness as a function of the team rather than of individuals. 

Hypothesis 3 was confirmed. A modest, positive correlation was found between 
TMPAS, the student's self-rating, and UCRSl, the student's evaluation of the course 
(.2052, p.=.0127). Evidently, the student feels that the team process is tied to the 
effectiveness of the course as a whole. 

Hypothesis 4 was confirmed. A modest positive correlation was found between 
TMPAS, the student's self-rating , and TARGET, the student's evaluation of the 
teacher's motivational decisions (r=.2594, p.=0015). This suggests that the student 
perceives the teams effectiveness to be involved with motivational structure set up by 
the instructor. 



Hypothesis 5 was confirmed. Positive correlations were found between TMPAS, the 
student's self-rating, and QTOT, the grand total of all the student's quiz scores 
(r=-2113, p.=.0113); and FIN, the student's final examination score (r=.2707, p.=.0009). 
While the correlation of TMPAS with MID, the student's score on the midterm 
examination, was positive, it did not quite reach significance (r=.1562, p.=.0580). This 




8 



6 



relationship to achievement is, of course, the bottom line. One could wish that the 
correlation were a great deal stronger. 

Educational Importance 

Of course, correlation does not indicate causation. It could be that a student's level of 
success with the team and the course influences his/her self-evaluation as an effective 
team member. It could also be that this self-concept influences the student's 
achievement in and appreciation for the instructor/ course. Some researchers suggest 
that it probably works both ways (Marsh, 1987; Shavelson & Bolus, 1982). 

Hoge, Smit, and Hanson (1990) found that teacher feedback and evaluation has a 
significant influence on student self-esteem. It would seem that an instructor should 
pay special attention to how STAD teams function, and provide accurate feedback and 
evaluation (such as the TMPA instrument) to the members. 
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Table 1 



Rotated Factor Pattern for UCRS2 



Item 


Factor 1 


Factor 2 


Factor 3 


A 


0.07836 


0.69193 


0.12490 


B 


0.36320 


0.58192 


-0.30593 


C 


0.19670 


0.58731 


-0.40395 


D 


0.77268 


0.25255 


0.06187 


E 


0.27006 


0.71172 


0.12786 


F 


0.10361 


0.23254 


0.79150 


G 


0.60411 


0.45104 


-0.18749 


H 


0.69302 


0.41617 


-0.00718 


I 


0.03367 


0.76926 


0.19525 


J 


0.64069 


0.39456 


-0.32949 


K 


0.73521 


-0.09374 


-0.00112 


L 


0.69043 


0.07623 


0.10489 




Percent of Variance Explained by each Factor 




Factor 1 . 


Factor 2 


Factoid 




31.31526 


29.25415 


11.11909 
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Table 2 



Pearson Product Moment Correlations Between TMPAS and 
Other Variables for 154 Educational Psychology Students 



MID TARGFT UCRS TMPAI TMPA2 QTQT EBl. 

PAS 

r= .1562 .2593 .2052 .5279 .4296 .2113 .2707 

p. = .0580 .0015 .0127 .0001 .0001 .0113 .0009 
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Section 



Teanv SS# 

UNlVERSITVCCUMgE RATING SCALE 



■ llarn°*^m ®'®^’’ what I was expected to 

almost alvvays^^ ^ (3) sometimes. (4) frequently. (5) 

P’’°i®®*®- papers, etc., helped me fulfill the course 
skills M \ ha rHK/^ assisted me in mastering the appropriate materials and 
almost alway^ ^ ^ eecasionally, (3) sometimes. (4) frequently. (5) 

II?f textbook and/or other assigned readings helped me fulfill the course 
^kiifa^ 'y?f assisted me in mastering the appropriate materials and 
almost alway^^^^ ^ occasionally. (3) sometimes. (4) frequently. (5) 



D. The in-class activities planned by the instructor helped me fulfill the course 
objectives. The experiences I had in class assisted me in mastering the 
course materials and skills. (1) hardly ever. (2) occasionally. (3) sometimes 
(4) frequently, (5) almost always. 



E. The measurement devices used in this course were accurate indicators of the 
extent to which I was fulfilling the objectives. They allowed me to show what I 
learned. 91) hardly ever. (2) occasionally. (3) sometimes. (4) frequently (5) 
almost always. ^ 



F. The feedback m this course was adequate. During the semester. I knew how 
well I was doing. (1) hardly ever. (2) occasionally. (3) sometimes (4) 
frequently. (5) almost always. 

G. My interest in the course was kept high enough to motivate me to do good 

work. I was able to apply myself. (1) hardly ever, (2) occasionally (3) 
sometimes. (4) frequently. (5) almost always. J'- ' / . 

H. In my opiriion, the teaching of this course was: (1) poor. (2) fair. (3) average 

(4) good, (5) excellent. ^ 

I. The grades I received for this course were fair. They were what I deserved 
for what I learned. (1) hardly ever, (2) occasionally, (3) sometimes (4) 
frequently. (5) almost always. 

J. Compared with other courses I have taken at this institution, I feel that I 
learned; (1) much less than in most courses, (2) less than most, 3) about an 
average amount, (4) more than in most, (5) much more than in most. 

K. Compared with other courses I have taken in this institution, I feel that the 
level of performance expected of me was: (1) much lower than most, (2) 
lower than most, (3) about the same as most, (4) higher than most, (5) much 
higher than most. 

.. The activities carried out in class helped me fulfill the course objectives. 
My team helped me master the course materials and skills. (1) hardly ever 
(2) occasionally, (3) sometimes, (4) frequently, (5) almost always. 



SecUon: Team: Social Security Humben 



TEAM MEMBER PERFORMANCE APPRAISAL 



DIRECTIONS: In the space provided, list in alphabetical order by last name, all members of 
your t«.aTn including yourself. Rate the contribution of each member on all of the five catena 
listed below. Put an asterisk (*) in front of your name. 

A. Attends team meetii^. 

B. Contributes helphil ideas to team discussions. 

C. Completes team assignments on time. 

D. Promotes positive feelings among team members. 

E. Encourages other team members to express themselves. 

Using the following 5-point scale, rate each criterion for each team member. 

5 = almost always 
4 = often 
3 = sometimes 
2 = seldom 
1 = almost never 



RATINOS OF CRITERIA 



TEAM MEMBER NAMES A B C D E SUM. OF A THRU E 



1. 














2. 














3. 














4. 














5. 














6. 














7. 
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Section# Team# SS# 

TARGETT MODEL RATING SCALE 

Evaluate this course using the TARGETT model summarized on Table 10.1 on page 369 
of your text. It has been reproduced on the back of this sheet. Rate each of the seven 
areas on a five point scale: 5 = excellent; 4 = good; 3 = average; 2 = poor; 1 = 

unacceptable. In the space provided, explain each rating, referring to the objectives 
and examples provided on the table. 

Task 



Autonomy/Responsibility 



Recognition 



Grouping 



Evaluation 



Time 



Teacher Expectations 




Total Rating (Sum the six area ratings.) 



15 






table 10.1 T b" TARGETT Model for. Supportin g Student Mbdvaribn to Leam 

Teachers make decisions in many areas that can influence motivation to learn. The TARGETT acronym highlights 
task, autonomy, recognition, grouping, evaluation, time, and teacher expectations. 



TARGETT Area 



Focus 



Objectives 



Examples of Possible Strategies 



Task 



How learning tasks are 
structured— what 
the student is asked 
to do 



Enhance intrinsic attractive- 
ness of learning tasks 
Make learning meaningful 



Encourage instruction riiat relates to students' 
backgrounds and experience 
Avoid payment (monetary or other) for 
attendance, grades, or achievcmCTt 






Autonomy/ 

Responsibility 



Recognition 



Grouping 



Evaluation 



Time 



Teacher 

Expeaations 



Student participation 
in leaming/school 
decisions 



The nature and use of 
recognition and 
reward in the school 
setting 



The organization of 
school learning and 
experiences 



The nature and use of 
evaluation and 
assessment 
procedures 



The scheduling of the 
school day 



Beliefs and predictions 
about students’ 
abilities 



Provide optimal freedom for 
students to make choices 
and take responsibility 



Provide opportunities for all 
students to be recognized 
for learning 

Recognize progress in goal 
attainment 

Recognize challenge seeking 
and innovation 

Build an environment of 
aocepunce and appre- 
dation of all students 

Broaden the range of social 
interaction, panicularly of 
at-risk students 

Enhance social skills 
development 

Grading and reporting 
processes 

Practices associated with use 
of standardized tests 

Definition of goals and 
standards 



Allow the learning task and 
student needs to dictate 
scheduling 

Provide oppominiries for 
extended and significant 
student involvement in 
learning tasks 

Hold appropriate but high 
expectations for all 
students 

Communicate that you 
expea growth 



Give alternatives in making assignments 
Ask for student comments on school life — 
and take them seriously 
Encourage students to take initiatives and 
evaluate their own learning 
Establish leadership opportunities for all 
students 

Foster “personal best" awards 
Reduce emphasis on “honor rolls" 
Recognize and publicize a wide range of 
school-related activities of students 



Provide opportunities for cooperative 
learning, problem solving, and decision 
making 

Encourage multiple group membership to 
increase range of peer interaction 
Eliminate ability-grouped classes 



Reduce emphasis on social comparisons of 
achievement 

Give students opportunities to improve their 
performance (e.g^ study skills, classes) 

Establish grading/reporring practice that 
portray student progress in learning 

Encourage student participation in the 
evaluation process 

Allow students to progress at their oum rate 
whenever possible 

Encourage flexibility in the scheduling of 
learning experiences 

Give teachers greater control over time usage 
through, for example, block scheduling 



Give all studenc the chance to revise and 
improve their work 

Monitor who gets which opportunities 
Make sure materials show diversity in 
achievement 



ERIC 



Source- M L. Maehr and E. M. Andermann (1993). Reinventing schools for early adolescent: to task goaU. The Elementary School 

Journal, 93. pp. 604-^05. Published by The University of Chicago Press. Adapted by permission. 
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